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Abstract In this paper, we address the problem of minimizing a convex 
function / over a convex set, with the extra constraint that some variables 
must be integer. This problem, even when / is a piecewise linear function, 
is NP-hard. We study an algorithmic approach to this problem, postponing 
its hardness to the realization of an oracle. If this oracle can be realized in 
polynomial time, then the problem can be solved in polynomial time as well. 
For problems with two integer variables, we show that the oracle can be 
implemented efficiently, that is, in 0(hi(B)) approximate minimizations of 
/ over the continuous variables, where B is a known bound on the absolute 
value of the integer variables. Our algorithm can be adapted to find the 
second best point of a purely integer convex optimization problem in two 
dimensions, and more generally its fc-th best point. This observation allows 
us to formulate a finite-time algorithm for mixed-integer convex optimization. 



1 Introduction 

One of the highlights in the list of publications of Martin Grotschel is his joint 
book with Laszlo Lovasz and Alexander Schrijver on Geometric Algorithms 
and Combinatorial Optimization [GLS88]. This book develops a beautiful 
and general theory of optimization over (integer) points in convex sets. The 
generality comes from the fact that the convex sets under consideration are 
presented by oracles (membership, separation in different variations, opti- 
mization) . The algorithms and their efficiency typically depend on the oracle 
presentation of the underlying convex set. This is precisely the theme of this 
paper as well: we present an algorithmic framework for solving mixed-integer 
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convex optimization problems that is based on an oracle. Whenever the oracle 
can be realized efficiently, then the overall running time of the optimization 
algorithm is efficient as well. 

One of the results from the book [GLS88] that is perhaps closest to our 
results is the following. Here and throughout the paper B(p, r) denotes a ball 
of radius r with center p. 

Theorem 1. [GLS88, Theorem 6.7.10] Let n be a fixed integer and K C R" 
be any convex set given by a weak separation oracle and for which there exist 
r, R > and p e K with B(p,r) C K C B(0,R). There exists an oracle- 
polynomial algorithm that, for every fixed e > 0, either finds an integral point 
in K + B(0, e) or concludes that K n Z™ = 0. 

The main distinction between results presented here and results from 
[GLS88] of such flavor as Theorem 1 results from dropping the assumption 
that we know a ball B(p,r) C K. At first glance, this might sound harmless, 
but it is not since the proof of Theorem 1 in [GLS88] uses a combination of 
the ellipsoid algorithm [Kha79] and a Lenstra-type algorithm [Len83]. In fact, 
dropping the assumption to know a ball B(p, r) C K requires new algorithmic 
frameworks. 

Let us now make precise our assumptions. We study a general mixed- 
integer convex optimization problem of the kind 

min{/(x, y) :(x,y)eSn (Z n x R d )}, (1) 

where the function / : M. n+d — > R+ U {+00} is a nonnegative proper con- 
vex function, i.e., there is a point z <G M. n+d with f(z) < +00. Moreover, 
S C M. n+d is a convex set that is defined by a finite number of convex func- 
tional constraints, i.e., S := {(x, y) e R n+d : gi(x, y) < for 1 < i < m}. We 
denote by (•, •) a scalar product. The functions g { : « n+d ->■ R are differen- 
tiable convex functions and encoded by a so-called first-order oracle. Given 
any point (x , yo) & R" +d , this oracle returns, for every i £ {1, . . . , m}, the 
function value gi(xo,yo) together with a subgradient g-(xo, yo), that is, a 
vector satisfying: 

9i(x,y) - 9i(x ,yo) > {g'i(x ,yo),(x - x ,y - y )) 

for all {x, y) G R n+d . 

In this general setting, very few algorithmic frameworks exist. The most 
commonly used one is "outer approximation" , originally proposed in [DG86] 
and later on refined in [VG90, FL94, BBC + 08]. This scheme is known to be 
finitely converging, yet there is no analysis regarding the number of iterations 
it takes to solve problem (1) up to a certain given accuracy. 

In this paper we present oracle-polynomial algorithmic schemes that are 
(i) amenable to an analysis and (ii) finite for any mixed-integer convex opti- 
mization problem. Our schemes also give rise to the fastest algorithm so far 



Mirror-Descent Methods in Mixed-Integer Convex Optimization 



3 



for solving mixed-integer convex optimization problems in variable dimension 
with at most two integer variables. 



2 An algorithm based on an "improvement oracle" 

We study in this paper an algorithmic approach to solve (1), postponing its 
hardness to the realization of an improvement oracle defined below. If this 
oracle can be realized in polynomial time, then the problem can be solved 
in polynomial time as well. An oracle of this type has already been used in 
a number of algorithms in other contexts, such as in [AK07] for semidefinite 
problems. 

Definition 1 (Improvement Oracle). Let a, 5 > 0. For every z <E S, the 
oracle 

a. returns z e S n (Z n x R d ) such that f(z) < (1 + a)f(z) + S, and/or 

b. asserts correctly that there is no point z e S n (Z™ x M. d ) for which 

m < /(*)■ 

We denote the query to this oracle at z by O a ,s(z). 

As stressed in the above definition, the oracle might content itself with a 
feasible point z satisfying the inequality in a without addressing the problem 
in b. However, we do not exclude the possibility of having an oracle that can 
occasionally report both facts. In that case, the point z that it outputs for 
the input point z£S must satisfy: 

f{z) - f* < af(z) + S + (f(z) - /*) < af(z) + S < af* + 6, 

where /* is the optimal objective value of (1). Thus f(z) < (1 + ce)f(z) + S, 
and it is not possible to hope for a better point of S from the oracle. We can 
therefore interrupt the computations and output z as the final result of our 
method. 

In the case where /* > and S = 0, the improvement oracle might be 
realized by a relaxation of the problem of finding a suitable z: in numerous 
cases, these relaxations come with a guaranteed value of a. In general, the 
realization of this oracle might need to solve a problem as difficult as the 
original mixed-integer convex instance, especially when a = S = 0. Never- 
theless, we will point out several situations where this oracle can actually be 
realized quite efficiently, even with a = 0. 

The domain of /, denoted by dom/, is the set of all the points z £ M. n+d 
with f(z) < +oo. For all z G dom/, we denote by f'(z) an element of the 
subdifferential df(z) of /. We represent by z* — (x*,y*) a minimizer of (1), 
and set /* := f(z*); more generally, we use a hat ( r ) to designate vectors 
that have their n first components integral by definition or by construction. 



4 



Michel Bacs, Timm Oertel, Christian Wagner, and Robert Wcismantcl 



Let us describe an elementary method for solving Lipschitz continuous 
convex problems on S approximately. Lipschitz continuity of / on S, an 
assumption we make from now on, entails that, given a norm || • || on R™ +d , 
there exists a constant L > for which: 

\f(z 1 )-f(z 2 )\<L\\z 1 -z 2 \\ 

for every Zi,z 2 € S. Equivalently, if || • ||» is the dual norm of || • ||, we have 
ll/'(z)ll* < L f° r every f'(z) € df(z) and every z <G dom/. 

Our first algorithm is a variant of the well-known Mirror-Descent Method 
(see Chapter 3 of [NY83]). It requires a termination procedure, which used 
alone constitutes our second algorithm as a minimization algorithm on its 
own. However, the second algorithm requires as input an information that 
is a priori not obvious to get: a point z e S for which f(z) is a (strictly) 
positive lower bound of /* . 

Let V : M. n+d — > R + be a diffcrcntiable er-strongly convex function with 
respect to the norm || • ||, i.e., there exists a a > for which, for every 
zi, z 2 <G M™ +d , we have: 

V(z 2 )-V(z 1 )-(V'(z 1 ),z 2 -z 1 ) > |||z 2 - Zl || 2 . 

We also use the conjugate T4 of V defined by V*(s) := sup{(s,z) — V(z) : 
z e R n+d } for every s G K n+d . We fix zq e S as the starting point of our 
algorithm and denote by M an upper bound of V(z*). We assume that the 
solution of the problem sup{(s,z) — V(z) : z e R rl+d } exists and can be 
computed easily, as well as the function p(w) := min{||w — z\\ : z € S} for 
every w € K" +d , its subgradient, and the minimizer tt(w). In an alternative 
version of the algorithm we are about to describe, we can merely assume that 
the problem max{(s, z) — V(z) : z € S} can be solved efficiently 

A possible building block for constructing an algorithm to solve (1) is the 
continuous optimum of the problem, that is, the minimizer of (1) without 
the integrality constraints. The following algorithm is essentially a standard 
procedure meant to compute an approximation of this continuous minimizer, 
lined with our oracle that constructs simultaneously a sequence of mixed- 
integer feasible points following the decrease of /. Except in the rare case 
when we produce a provably suitable solution to our problem, this algorithm 
provides a point z e S such that f(z) is a lower bound of /*. Would this 
lower bound be readily available, we can jump immediately to the termination 
procedure (see Algorithm 2). 

The following proposition is an extension of the standard proof of conver- 
gence of Mirror-Descent Methods. We include it here for the sake of com- 
pleteness. 

Proposition 1. Suppose that the oracle reports a for k = 0, . . ., N in Algo- 
rithm 1, that is, it delivers an output ik for every iteration k — 0,...,N. 
Then: 
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Data: zq g S. 

Set zo := zo, wo := zo, so := 0, and fo := /(zo). 
Select sequences {h k } k > , {a k } k > , {8 k } k > . 
for k = 0, . . . , N do 

Compute /'(z fc ) € <9/(z fe ) and p'(w k ) G dp(w k ). 

Set s fc+1 := s fc - h k f'(z k ) - h k \\f'(z k )\\ r p'(w k ). 

Set := argmax{{s t+1 , z) - V(z) : z G M n+d }. 

Set z k+1 := argmin{||u> fc+1 - z\\ : z e S}. 

Compute f(z k+1 ). 

if /Ofc+i) > fk then z fc+1 := z k , f k+1 := f k . 
else 

Run O ak+l! s k+1 (z k+1 ). 

if f/ie oracle reports a and b then 

j Terminate the algorithm and return the oracle output from a. 
else if the oracle reports a but not b then 

i Set z k+ i as the oracle output and /fc+i := min{/(zfc + i), f k }. 
else 

Run the termination procedure with zo := Zfc+i, zo := Zfc+l: 
return its output, and terminate the algorithm. 

end 

end 

end 

Algorithm 1: Mirror-Descent Method. 

Data: z £ 5 with f(z ) < /*, zo £ S D (Z n X M d ). 
Set l := /(zo), «0 := /(zo)- 

Choose a, 5 > 0. Choose a subproblem accuracy e' > 0. 
for > do 

Compute using a bisection method a point z k +i = Xz k + (1 — X)z k 
for < A < 1, for which f(z k+1 ) - (J fc (a + 1) + u k )/(a + 2) e [-e' : e']. 
Run Qjl5 (z fe+ i). 

if (fee oracle reports a and b then 

| Terminate the algorithm and return the oracle output from a. 
else if the oracle reports a but not b then 

I Set z k+1 as the oracle output, l k+1 := l k , u k+1 := min{/(z fc+1 ), u k }. 
else 

| Set z k + 1 := z k , l k + 1 := f(z k + 1 ), u k+1 := u k . 
end 

end 

Algorithm 2: Termination procedure. 



1 f^ h k f(z k ) M 21? Eto^t 1 y> hkh 

Proof. Since is cr-strongly convex with respect to the norm || • ||, its conju- 
gate V* is differentiable and has a Lipschitz continuous gradient of constant 
l/cr for the norm || • ||«, i.e., V*(y) - V*(x) < (V*(x),y - x) + ^\\y - x\\l 
(see [HUL93, Chapter X]). Also w k = Vl{s k ), in view of [Roc81, Theo- 
rem 23.5]. Finally, for every z G S, we can write p(w k ) + (p'(wk), z — w k ) < 
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p(z) = 0. Thus: 

(p'(w k ),w k - z*) > p(w k ) = ||7r(t0fe) - w k \\ = \\z k - w k \\. (2) 

Also, ||p'(wfc)||* < 1, because for every z e W n+d : 

(p'(w k ),z- w k ) < p(z) - p(w k ) = \\z - n(z)\\ - \\w k - 7r(iUfc)|| 

< \\z - n(w k )\\ -\\w k - n(w k )\\ < \\z - w k \\. (3) 

By setting (f> k := V*(s k ) — (s k , z*), we can write successively for all k > 0: 
4>k+i = V*(s k +i) - (s k+ i,z*) 



< V*(sfe) + Sfc+l - Sfe) + 7^-||Sfe+l 

la 



Sk\\t - (Sk+l,Z*). 



= (K(sfe) - (s k ,z*)) + (K'( s fe) - z*,s k+ i - s k ) + — \\s k +i - Sfe | 

la 



4> k - h k (w k - z k ,f'(z k )) + h k (z* - z k , f'(z k )) 



-h k \\f'{z k )\\* (w k - z*,p'{w k )) + 



hll\f'{z k )l\] 
2a 



f(z k ) 



\\f'(z k )\l 



+ p'(w k ) 



where the inequality follows from the Lipschitz continuity of the gradient 
of V*, and the last equality from the identities Vl{s k ) — w k , s k+ i — s k = 
-h k f'(z k ) - h k \\f'(z k )\\*p'(w k ), and V*(sfe) - (s k , z*) = <j> k . By the definition 
of the dual norm, it holds -h k (w k - z k ,f'(z k )) < h k \\f'(z k )\\*\\w k - z k \\. 
Moreover, convexity of / implies h k (z* — z k ,f'(z k )) < f(z*) — f(z k ). Using 
this in the above expression we get: 



4>k+i < 4>k + h k \\f'(z k )\\* (\\w k - Zfe|| - (w k - z*,p'(w k ))) 

hl\\f'(zk)\\l ( f'(z k ) 



+h k (f(z*) - f(z k )) + 

< 4>k + hk(f(z*)- 
<<t>k + h k (f(z*) 



2a 



\\f'(z k )\U 



\\p'(wk)L 



f(zk)) + ~ k 
f(z k ) - 5 k 



2hl\\f'(z k )\\l 



1 + a k 



+ 



2hl\\f'(z k )\\l 



where the second inequality follows from (2) and ||p'(wfe)||* < 1, and the 
third inequality from the fact that the oracle reports a. Summing up the 
above inequalities from k := to k :— N and rearranging, it follows: 



N 



Z^k=o nk k=o 



h k (f(z k ) - S k ) 
1 + a k 



<j> -<f>N+i , 2Ef =0 ^ll/'(^)ll 



-nn< 



a EfeLo h k 



Note that ||/'(^fe)||* < L, (p Q = sup{-V(z) : z G 
— V(z*~) > —M, yielding the desired result. 



< 0, and 



>N+1 



> 
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In the special case when a k — a and S k = S for every k > 0, we can 
significantly simplify the above results. According to the previous proposition, 
we know that: 

I m.mi<j< N f(zj) - 5 _ 
1 + a 





k=a Vfe=o / fe=o 

We can divide both sides of the above inequality by J2k=o ^ fc > then determine 
the step-sizes {hk ■ < k < N} for which the right-hand side is minimized. 
However, with this strategy, ho would depend on N, which is a priori unknown 
at the first iteration. Instead, as in [Nes04], we use a step-size of the form 
hk = c/y/k + 1 for an appropriate constant c > 0, independent of N. Note 
that: 

^ i v^i r N+1 dt , WAr , 

If we choose c := \J the right-hand side of (4) can be upper-bounded by 
Mln(N + 1) + 2M. Finally, since 

we can thereby conclude that: 

,f N - (1 + a)f* - S < £ /M ln(jV + l) + 2 

1 + a - V 2cr ' VFT2- 1 U 

As the right-hand side converges to when N goes to infinity, Algorithm 1 
converges to an acceptable approximate solution or calls the termination 
procedure. 

Let us now turn our attention to the termination procedure. We assume 
here that the oracle achieves a constant quality, that is, that there exists 
a, S > for which ak — a and 5k — S for every k > 0. 

Proposition 2. Assume that f(zo) > f{zo) > 0, and that there is no point 
ieSfl (Z n x R d ) for which f(z Q ) > f(z). 

(a) The termination procedure cannot guarantee an accuracy better than: 

f{z) < f* + (2 + a) (af* + (1 + a)e' + s) . (6) 
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(b) For every e > 0, the termination procedure finds a point z e S(~) {IT 1 x R d ) 
satisfying: 

f{z) - f* < ef + (2 + a) (af* + (1 + a)e' + 

within 

max 

iterations. 

Proof. Part (a). At every iteration k, there is by construction no z G SO 
(II 1 x R d ) for which Ik > f(z). Also, f(zk) > Uk > /*. For convenience, we 
denote (1 + a)/ (2 + a) by A in this proof, and we set A}. := Uk — h for every 
k > 0. 

Suppose first that the oracle finds a new point Zk+i € S n (Z n x R d ) at 
iteration fc. Then: 

f(z k +i) < (1 + a)/(^ fe+ i) + 5 < (1 + a) (A; fe + (f - AH + + 5, 

where the first inequality is due to the definition of our oracle and the second 
one conies from the accuracy by which our bisection procedure computes 
Zk+i- Observe that the oracle might return a point Zk such that f(zk) is 
smaller than the above right-hand side. In this case, no progress is done. As 
u k < f{zk), this implies: 

(A + Xa)l k + (1 + a)e' + 6>(\ + Xa- a)f{z k ). (7) 

Using that /* > Ik we get an upper bound of the left-hand side. Rearranging 
the terms and replacing A by its value, we get: 

f + (2 + a)(af* + (1 + a)e' + 5)> f(z k ). 

Since all the inequalities in the above derivation can be tight, a better accu- 
racy cannot be guaranteed with our strategy. Thus, we can output z k - 
Part (b). Note that we can assume ^ z jl|~^ Z( ^ > 1, for otherwise the point 
Zo already satisfies our stopping criterion. 

In order to assess the progress of the algorithm, we can assume that the 
stopping criterion (7) is not satisfied. As lk+i = Ik in our case where the 
oracle gives an output, we get: 
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A k +i — Wfc+i ~h< f(zk+i) ~ h 

< (1 + a) (\l k + (1 - AH +e')+5-l k 




Suppose now that the oracle informs us that there is no mixed-integral 
point with a value smaller than f(zk+i) > AZfc + (l — X)u k — e'. Then z k +i = z k 
and Uk+i = Uk- We have: 



The above inequality is valid for every k that does not comply with the 
stopping criterion, whatever the oracle detects. Therefore, we get: 



In the remainder of this paper, we elaborate on possible realizations of our 
hard oracle. 

We proceed as follows. In Section 3, we focus on the special case when 
n = 2 and d — 0. We present a geometric construction that enables us to 
implement the improvement oracle in polynomial time. With the help of this 
oracle we then solve the problem (1) with n = 2 and d = and obtain a 
"best point", i.e., an optimal point. An adaptation of this construction can 
also be used to determine a second and, more generally, a "fc-th best point" . 
These results will be extended in Section 4 to the mixed-integer case with 
two integer variables and d continuous variables. The latter extensions are 
then used as a subroutine to solve the general problem (1) with arbitrary n 
and d in finite time. 



3 Two-dimensional integer convex optimization 

If n = 1 and d = 0, an improvement oracle can be trivially realized 
for a = S = 0. Queried on a point z e M the oracle returns z := 



Ak+i = Mfe+i - h+i = f{zk) - f(zk+i) 

<u k - (A/ fe + (1 - \)u k - e) = \A k + e 



< l±^A k + a r + (l + a)e' + 6. 
2 + a 




and the proposition is proved because J{zn) — f* < ^W- 



□ 
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argmin{/(|_2_|), f(\z] )} if one of these numbers is smaller or equal to f(z), or 
returns b otherwise. The first non-trivial case arises when n — 2 and d = 0. 
This is the topic of this section. 



3. 1 Minimizing a convex function in two integer 
variables 

We show in this section how to implement efficiently the oracle O a .s with 
a = 5 = 0, provided that the feasible set is contained in a known finite box 
[-B,B]\ 

Theorem 2. Let f : E 2 — > E and g { : M 2 — > M loiift i = 1, . . . , m &e convex 
functions. Let Bel and ZeZ a; G [— -B,-B] 2 suc/i ZZiaZ <7»(ar) < /or aZZ 
i = 1, . . . , m. T/ien, m a number of evaluations of f and gi,...,g m that is 
polynomial in ln(_B), one can either 

(a) find an x G [-B,B] 2 n Z 2 with f(x) < f(x) and g^x) < for all i = 
1, . . . , m or 

(b) show that there is no such point. 

Note that we do not allow for the function / to take infinite values, in 
order to ensure that we can minimize / over the integers of any segment of 
[-B, B} 2 in C(ln(_B)) evaluations of / using a bisection method. Indeed, if a 
convex function takes infinite values, it can cost up to 0{B) evaluations of / 
to minimize it on a segment containing 0(B) integer points, as there could 
be only one of those points on its domain. 

The algorithm that achieves the performance claimed in Theorem 2 is 
described in the proof of the theorem. That proof requires two lemmata. We 
use the following notation. Let Qct 2 . We denote by vol(Q) the volume of 
Q, i.e., its Lebesgue measure. By aff{(5} we denote the smallest affine space 
containing Q and by conv{Q} the convex hull of Q. The dimension dim(Q) 
of Q is the dimension of aff{Q}. The scalar product we use in this section is 
exclusively the standard dot product. 

Lemma 1. Let ifcM 2 be a polytope with vol(K) < |. Then dim(conv(ii' fl 
I?)) < 1. 

Proof. For the purpose of deriving a contradiction, assume that there ex- 
ist three affinely independent points x,y,z G K n 1? . Then vol(K) > 
vol(conv({x, y, z})) — \\ det(x — z, y — z)\ > |. □ 

Lemma 2. Let u, v, w G E 2 be affinely independent. If 

(conv{u, u + v,u + v + w}\ (conv{u + v, u + v + w} U {u}) ) n Z 2 = 0, 
then the lattice points conv{u, u + v, u + v — w} (1 1? lie on at most three lines. 
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Proof. We partition conv{u, u + v, u + v — w} into three regions. Then we 
show that in each region the integer points must lie on a single line using a 
lattice covering argument. 

We define the parallelogram P := conv{0, \v, ^w, \v + \w}. Further, we 

set 

Ai\=u— ^w + P, A 2 := u + — w + P, and A 3 := u + — + P. 

Note that conv{u , u+v,u+v — w} C A 1 UA 2 UA 3 (see Fig. 1 ) . Our assumption 
implies that the set u + \v + P does not contain any integer point except 
possibly on the segment u, + w + conv{0, w}. Therefore, for a sufficiently small 
e > 0, the set (u + \v — e(v + w) + P) n Z 2 is empty. 

Assume now that one of the three regions, say A\, contains three affinely 
independent integer points x,y,z. We show below that A\ + Z 2 = M 2 , i.e., 
that P defines a lattice covering, or equivalently that the set t + P contains 
at least one integer point for every i e K 2 . This fact will contradict that 
(u + \v — e(v + w) + P) n I? = and thereby prove the lemma. 
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Clearly, the parallelogram Q := conv{a;, y,z, x — y + z} defines a lattice 
covering, as it is full-dimensional and its vertices are integral. We transform 
Q into a set Q' C Ai for which a € Q' iff there exists b e Q such that 
a-kZ 2 . Specifically, we define a mapping T such that Q' = T(Q) C Ai and 
T(Q) + Z 2 = R 2 . Let v 1 - := (-v 2 ,vi) T and w 1 - := (-w 2 ,wi) T , i.e., vectors 
orthogonal to v and w. Without loss of generality (up to a permutation 
of the names x,y,z), we can assume that (x, w ) < (y,w ) < (z,w ). If 
x — y + z € A\ there is nothing to show, so we suppose that x — y + z ^ A\ . 

Note that (x, w 1 -) < (x — y + z, w 1 ) < (5, w 1 ). Assume first that (x — y + 
z,v^) < {z,v^-) < (x,v ± ), (y, v- 1 ) — the strict inequality resulting from the 
fact that x — y + z ^ A\. We define the mapping T : Q — > A\ as follows, 



T{1) = I 



l + y — z, if {I, v- 1 ) < (z, v^) and {I, w 1 ) > (x — y + z, 
I — x + y, if {I, y- 1 ) < (z, v- 1 -) and {I, w 1 ) < (x — y + z, 
I, otherwise 



(see Fig. 2). It is straightforward to show that T(Q) C Ax and T(Q) + 1? = 
M 2 . A similar construction can easily be defined for any possible ordering of 

(x — y + z,v ), (zjV- 1 ), (x, v ), and (y, v ). □ 

Remark 1. In each region Ai, the line containing Ai fl Z 2 , if it exists, can be 
computed by the minimization of an arbitrary linear function x M> (c, x) over 
Ai f] 7j 2 , with c 7^ 0, and the maximization of the same function with the 
fast algorithm described in [EL05] . If these problems are feasible and yield 
two distinct solutions, the line we are looking for is the one joining these two 
solutions. If the two solutions coincide, that line is the one orthogonal to c 
passing through that point. 

The algorithm in [EL05] is applicable to integer linear programs with two 
variables and m constraints. The data of the problem should be integral. This 
algorithm runs in 0(m + </>), where <j> is the binary encoding length of the 
data. o 

Proof (of Theorem 2). As described at the beginning of this subsection, 
a one-dimensional integer minimization problem can be solved polynomi- 
ally with respect to the logarithm of the length of the segment that the 
function is optimized over. In the following we explain how to reduce the 
implementation of the two-dimensional oracle to the task of solving one- 
dimensional integer minimization problems. For notational convenience, we 
define g(y) := max i= i... m gi(y) for y £ R 2 which is again a convex function. 

Let Fx, . . . ,F 4 be the facets of [-B, B} 2 . Then [-B, B} 2 = [f ]=1 conv{x, Fj}. 
The procedure we are about to describe has to be applied to every facet 
Fi, . . . , i<4 successively, until a suitable point x is found. Let us only consider 
one facet F. We define the triangle T := conv{x, F}, whose area is smaller 
than 2B 2 . 

To find an improving point within T , we construct a sequence T D T\ D 
T 2 D . . . of triangles that all have x as vertex, with vol(Tfc + i) < |vol(T/j), 
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and such that f(y) > f(x) or g(y) > for all y e ( T \ T k ) n I?. Wc stop 
our search if we have found an x € [-B,B] 2 flZ 2 such that f(x) < f(x) and 
g(x) < 0, or if the volume of one of the triangles T k is smaller than i. The 
latter happens after at most k = |~ln(4_B 2 )/ ln(|)] steps. Then, Lemma 1 
ensures that the integral points of T k are on a line, and we need at most 
0(ln(£>)) iterations to solve the resulting one-dimensional problem. 

The iterative construction is as follows. Let Tk — conv{x, vo, v\} be given. 
We write v\ := (1 — X)vo + Xvi for A € R and we define the auxiliary triangle 
Tk := conv{x, u 1 / 3 , U2/3}. Consider the integer linear program 

mm{(h,y) :yeT k nZ 2 } (8) 

where h is the normal vector to conv{w ,wi} such that (h,x) < (h,y) for 
every y <G F. We distinguish two cases. 

Case 1. The integer linear program (8) is infeasible. Then Tf. flZ 2 = 0. It 
remains to check for an improving point within (Tk\Tk)C\l? . By construction, 
we can apply Lemma 2 twice (with (u, u+v—w, u+v+w) equal to (x, vq, V2/3) 
and (x, V\/3,Vi), respectively) to determine whether there exists an x € (Tk \ 
T k ) n 1? such that f(x) < f(x) and g(x) < 0. This requires to solve at most 
six one-dimensional subproblcms. 

Case 2. The integer linear program (8) has an optimal solution z. If 
f{z) < f(x) and g(z) < 0, wc are done. So we assume that f(z) > f(x) or 
g(z) > 0. Define H := {y £ M 2 \ (h, y) — (h, z)}, that is, the line containing z 
that is parallel to convj^o, fi}, and denote by H + the closed half-space with 
boundary H that contains x. By definition of z, there is no integer point in 
f k n intH + . Further, let L := aff{x, z}. 




Fig. 3 Illustration of Case 2. 
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Due to the convexity of the set {y € R 2 | f(y) < f(x), g{y) < 0} and the 
fact that f(z) > f(x) or g(z) > 0, there exists a half-space L + with boundary 
L such that the possibly empty segment {y e H \ f(y) < f(x), g(y) < 0} 
lies in L + (see Fig. 3). By convexity of / and g, the set ((Tk \ H + ) \ L + ) 
(the lightgray region in Fig. 3) contains no point y for which f(y) < f(x) 
and g{y) < 0. It remains to check for an improving point within ((Tk n H+) \ 
L + )r\Z 2 . For that we apply again Lemma 2 on the triangle conv{z!/ 3 , z\,x} 
(the darkgray region in Fig. 3), with z±n = H n aff{x, W1/3} and z\ = H n 
aff{a;, t>i}. If none of the corresponding subproblems returns a suitable point 
.£ e Z 2 , wc know that Tk \ L + contains no improving integer point. Defining 
Tk+i := Tk H L + , we have by construction f(y) > f(x) or g(y) > for all 
y € {T k \ Tk+i) n I? and vol(T fe+1 ) < |vol(T fe ). 

It remains to determine the half-space L + . If g(z) > we just need to find 
a point y € H such that g(y) < g{z), or if f(z) > f(x), it suffices to find a 
point y e H such that f(y) < f(z). Finally, if we cannot find such a point y 
in either case, convexity implies that there is no suitable point in Tk \ H + ; 
another application of Lemma 2 then suffices to determine whether there is 
a suitable x in T k n H+ n 1? . □ 

The algorithm presented in the proof of Theorem 2 can be adapted to 
output a minimizcr x* of / over S n [-B, B} 2 n Z 2 , provided that we know 
in advance that the input point x satisfies f(x) < /*: it suffices to store and 
update the best value of / on integer points found so far. In this case the 
termination procedure is not necessary. 

Corollary 1. Let f : R 2 — > M and g t : R 2 — >• M i = 1, . . . , m &e convex 
functions. Let fieN and fe< x G [— -6,-B] 2 smc/i i/iai 5i(x) < /or a/i i = 
1, . . . , m. If f(x) < /*, i/ien, m a number of evaluations of f and gi,... ,g m 
that is polynomial in ln(£>), one can either 

(a) find an x e [— i?, B] 2 n Z 2 wii/i /(x) = /* and ffi(x) < for all i = 

1, . . . , m or 
f&j s/iow i/iai i/iere is no smc/i point. 

Note that line 30 in Algorithm 3 requires the application of Lemma 1. 
Lines 11, 20 and 24 require the application of Lemma 2. 

Remark 2 (Complexity). The following subroutines are used in Algorithm 3. 

Line 9 and applications of Lemma 2. A two-dimensional integer linear pro- 
gram solver for problems having at most four constraints, such as the one 
described in [EL05]. The size of the data describing each of these con- 
straints is in the order of the representation of the vector x as a rational 
number, which, in its standard truncated decimal representation, is in 
0(\n(B)). 

Line 30 and applications of Lemma 2. A solver for one-dimensional inte- 
ger convex optimization problems. At every iteration, we need to perform 
at most seven of them, for a cost of 0(\n(B)) at each time. 
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Data: x 6 [-B, B} 2 with f(x) < /* and g^x) < for alH = 1, . . . , m. 

1 Let Fi, . . . , F 4 be the facets of [-B, B] 2 . 

2 Set x* := and /* := +oo. 

3 for t = 1, . . . , 4 do 

4 Set F := Ft and define vo, vi S R™ such that F := conv{«o, fi}- 

5 Write ft for the vector normal to F pointing outwards [-B, B] 2 . 

6 Set T := conv{x, F} and k := 0. 

7 while vol(T fc ) > § do 

8 Set T fc := conv{x, D1/3, 1*2/3}, with := (1 ~ ^) v o + ^ v i- 

9 Solve (V) : min{(/i, j):jef t nZ 2 }. 
10 if (Case 1) (V) is infeasible, then 

; 1 Determine x := argmin{/(£) | 2 e (T fc \ f fe ) n 7L 2 with 3(2) < 0}. 

L2 if a; exists and f(x) < f* then Set x* := x and /* := /(£). 

13 else 

14 ( Case 2) Let 5 be an optimal solution of (P). 

15 Set H+ := {y eR 2 : (h, y) < (h, z)} and H := 8H+. 

16 Define the points v := aff {x, z} n F and Zi = H (1 conv{a;, i>i} for 
i = 0,1. 

1 7 Denote z A := (1 - A)zo + Azi for A e (0, 1). 

18 if < and there is a y 6 conv{zo,5} /or which f(y) < /(z) or 

19 g(z) > and there is a y £ conv{zrj,z} /or which g(y) < g(z) then 

20 Determine 
£ := argmin{/(z) | z 6 conv{x, Z1/3, zi} n Z 2 with g(z) < 0}. 

21 if a; exists and f(x) < /* then Set £* := x and /* := /(£). 

22 Set vi := v, Tfe+i := conv{a;, i>o, f}, and k := k + 1. 

23 else 

24 Determine 
£ := argmin{/(z) | z £ conv{x, zo, Z2/3} HZ 2 with g(z) < 0}. 
if x exists and f(x) < /* then Set x* := x and /* := f(x). 
Set vo := v, Tfc +1 := conv{x, v, vi}, and k := k + 1. 



25 
26 
27 
28 
29 
30 
31 

32 end 

33 if /* < 



end 

end 

end 

Determine x := argmin{/(z) | z G Tj. n \ 
if x exists and f(x) < /* then Set x* :- 



2 with g(z) < 0}. 
x and /* := /(£) 



co then Return x*. 
34 else Return "the problem is unfeasible". 

Algorithm 3: Minimization algorithm for 2D problems. 



Lines 18 and 19. Given a segment [a, b] and one of its points z, we need 
a device to determine which of the two regions [a, z] or [z, b] intersects a 
level set defined by / and g that does not contain z. This procedure has a 
complexity of C(ln(_B)) and only occurs in Case 2 above. o 
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3.2 Finding the k-th best point 

In this subsection we want to show how to find the fc-th best point, provided 
that the k — 1 best points are known. A slight variant of this problem will be 
used in Subsection 4.3 as a subroutine for the general mixed-integer convex 
problem. In the following, we describe the necessary extensions of the previous 
Algorithm 3. Let x\ := x* and define for k > 2: 

x* k :-argmin{/(x) \xe (S H [-B, B} 2 n Z 2 ) \ {x{, . . . ,x* k ^}} 

to be the k-th best point. Observe that, due to the convexity of / and 
gi, . . . ,g m , we can always assume that conv{x*, . . . ,xl_ 1 }r\Z 2 = {x*, . . . ,x* k _ 1 } 
for all k > 2. Although this observation appears plausible it is not completely 
trivial to achieve this algorithmically. 

Lemma 3. Let II j := {x\ 1 . . . ,x*} be the ordered j best points of our problem 
and Pj be the convex hull of Llj. Suppose that, for a given k > 2, we have 
Pk-i H Z 2 = LTk-i- Let x* k be a k-th best point. 

(a) If f(x* k ) > f* , we can replace the point x* k by a feasible k-th best point z k 
such that conv{i7fc_i, z k } HZ 2 = {7T fe _ 1 , z* k } in 0(1) operations. 

(b) If f(x* k ) = /*, and if we have at our disposal the v vertices of Pk-i 
ordered counterclockwise, we can construct such a point i k in 0(vhi(B)) 
operations. 

Proof. Part (a). Suppose first that f{x* k ) > /*, and assume that we cannot 
set i* k := x* h , that is, that there exists x e (PkC\I?)\PI k . Then x = J2i=i 
for some Xi > that sum up to 1. Note that < Afe < 1, because x ^ 
Pk-i U {x k } by assumption, and that f(x) > f{x* k ). We deduce: 

k 

o < f(x) f{x%) < - m)) < o. 

i=l 

Thus f(x) = f(x* k ). Let /:={«: A, > 0} and Qj := conv{x* :iel}, so that 
x e rclint Qj. Observe that |/| > 2 and that / is constant on Qi. Necessarily, 
Qi is a segment. Indeed, if it were a two-dimensional set, we could consider 
the restriction of / on the line I := aff{x^,x}: it is constant on the open 
interval I fl int Qi, but does not attain its minimum on it, contradicting the 
convexity of /. Let us now construct the point z k : it suffices to consider the 
closest point to x* k in aff{Q/} n Pk-i, say x*, and to take the integer point 
z k ^ Xj of conv{x*,x k } that is the closest to x* (see Fig. 4). 
Part (b). Suppose now that f(x*) = f* for every 1 < i < k, and define 



ivo = vt,yl,---,vt-i} Q iik-\ 
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Fig. 4 Illustration of Part (a). 



as the vertices of Pk-i, labeled counterclockwise. It is well-known that de- 
termining the convex hull of Pk-i U {x k } costs 0(ln(v)) operations. From 
these vertices we deduce the set {y* : i € J} of those points that are in 
the relative interior of that convex hull. Up to a renumbering of the yf's, 
we have J = {1,2, . . . ,j — 1}. We show below that Algorithm 4 constructs a 
satisfactory point zt. 

Let us denote by x k (i) the point x* k that the algorithm has at the beginning 
of iteration i and define Tj(i) := conv{a;£(i), \ ffe-i for < I < j 

(see Fig. 5). At iteration i, the algorithm considers the triangle Ti(i) if its 
signed area |det(x£(i) — yi,y* +1 — y*) is nonnegative, and finds a point 
x k (i + 1) G Ti(i) such that Ti(i + 1) has only x k (i + 1) as integer point. 

We prove by recursion on i that XJ(i) contains only x k (i) as integer point 
whenever I < i. We already noted it when % — 0. Suppose the statement is 
true for i, and let I < i. We have: 

i-l 

+ 1) e Ti(i) C conv{x^),y*, . . . \ P fe _! - Ti(i) U |J T^i), 

hence 

K := conv{x£(i + 1), . . . , y* +1 } \ P fc _i C conv{x£(i), $j, . . . , \ P fe _i. 

As Tj(i + 1) C X for all Z < i, the integers of T t (i + 1) are either in (Ji=o T K*) n 
Z 2 , which reduces to by recursion hypothesis, or in Tj(i). Since x£(i) € 

Ti(i), all the integers in T t (i + 1) must be in T t {i). But T ; (i + l)nT 4 (i)nZ 2 = 
+ 1)} by construction of x k (i + 1), and the recursion is proved. 

It remains to take the largest value that i attains in the course of Algo- 
rithm 4 to finish the proof. We need to solve at most v — 1 two-dimensional 
integer linear problems over triangles to compute x k ; the data of these prob- 
lems are integers bounded by B. □ 

By Lemma 3, the fc-th best point x* k can be assumed to be contained within 
[-B, B} 2 \ convjxj, . . . , xl_ 1 }. This property allows us to design a straight- 
forward algorithm to compute this point. We first construct an inequality 
description of conv{x|, . . . ,x k _ 1 }, say (di,x) < bi for i € / with |/| < +oo. 
Then 

[-B, B} 2 \ conv{xJ, . . . , = |J{x e [-B, B} 2 \ {a u x) > h}. 

iei 
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Data: x*, y£, y\, . . . , y* y 

Set i := and x^(0) := xt. 

while det(£*(i) - y*,y* +1 - y*) > do 

Set Ti :=conv{£*(i),y*,y* +1 }\aff{y*,j/* +1 }. 

Set hi a vector orthogonal to aS{y* , y* +1 } such that (hi,x^(i) — y*) > 0. 
Set + 1) := argmin{(fei,£) : x g Tj n Z 2 }. 
Set i := i + 1. 

end 

Set Si :=x%(i). 

Algorithm 4: A point z* k for which conv{IZfc_i, z\\ 01? = {i7fe-i, -2^}. 



Fig. 5 Constructing Pj. from Pfc_i. 

As the feasible set is described as a union of simple convex sets, we could 
apply Algorithm 1 once for each of them. However, instead of choosing this 
straightforward approach one can do better: one can avoid treating each ele- 
ment of this disjunction separately by modifying Algorithm 3 appropriately. 

Suppose first that k — 2. To find the second best point, we apply Algo- 
rithm 3 to the point x\ with the following minor modification: in Line 9, 
we replace (V) with the integer linear problem (V') : min{(/i, y) : y G 
f k n Z 2 , (h,y) > (h,xi) + 1}, where h e Z 2 such that gcd(h lt h 2 ) = 1. 
This prevents the algorithm from returning x\ again. 

Let k > 3. Let y^, . . . , yt-i,yt = denote the vertices of Pk-i, ordered 
counterclockwise (they can be determined in C(fcln(fc)) operations using the 
Graham Scan [Gra72]). Recall that the point we are looking for is not in 



Let us call a triangle with a point y* as vertex and with a segment of the 
boundary of [— B,B] 2 as opposite side a search triangle (see Fig. 7: every 
white triangle is a search triangle). The idea is to decompose [-B, B} 2 \ Pk-i 
into search triangles, then to apply Algorithm 3 to these triangles instead of 
(conv{a;,F t })^ =1 . 

For each < i < v, we define Hi := {y e M 2 : det(y - y*,y* +1 - y*) > 0}, 
so that Hi n Pfc_i = conv{y*,y* +1 }. Consider the regions Ri := ([—B, B] 2 n 
Hi) \ inti?,_i. Note that Ri contains only y* and y* +1 as vertices of Pk-i- 
Also, at most four of the i?,'s are no search triangles. If Ri is such, we 



.f-(.)) = ^.(l)^: 




p k -i- 
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triangulate it into (at least two) search triangles by inserting chords from y* 
to the appropriate vertices of [— B, B] 2 . 




Note that a search triangle can contain two or more integer points of 
Pk-i- In order to prevent us from outputting one of those, we need to 
perturb the search triangles slightly before using them in Algorithm 3. 
Let T = conv{y*, 61, 62} be one of the search triangles, with 61,62 being 
points of the boundary of [-B, B} 2 . The triangle T might contain y* +1 , say 
y* +1 <E conv{y*, &i}, a point we need to exclude from T. We modify bi slightly 
by replacing it with (1 — e)b\ + eb 2 for an appropriate positive e > whose 
encoding length is C(ln(_B)). 

So, we apply Algorithm 3 with all these modified search triangles instead 
of convja;, F\}, . . . , conv{x, F4}. A simple modification of Line 9 allows us to 
avoid the point y* for z: we just need to replace the linear integer problem (V) 
with mm{{h, y) : y € f k n Z 2 , (h, y) > (h, y*) + 1}, where h £ Z 2 such that 
gcd(/ii,/i 2 ) = 1. Then, among the feasible integer points found, we return 
the point with smallest objective value. 

Corollary 2. Let f : R 2 ->• R and g, : R 2 -> R with i = 1, . . . , m be convex 
functions. Let x*, . . . , x^_ 1 be the k — 1 best points for min{/(x) : x e S n 
[-B,B] 2 nZ 2 }. Then, in a number of evaluations of f and gi,... 7 g m that is 
polynomial in \n(B) and in k, one can either find 

(a) a k-th best point, x* k , or 

(b) show that there is no such point. 



4 Extensions and applications to the general setting 

In this section, we extend our algorithm for solving two-dimensional integer 
convex optimization problems in order to solve more general mixed-integer 
convex problems. The first extension concerns mixed-integer convex problems 
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with two integer variables and d continuous variables. For those, we first 
need results about problems with only one integer variable. We derive these 
results in Subsection 4.1 where we propose a variant of the well-known golden 
search method that deals with convex functions whose value is only known 
approximately. To the best of our knowledge, this variant is new. 

In Subsection 4.2, we build an efficient method for solving mixed-integer 
convex problems with two integer and d continuous variables and propose 
an extension of Corollary 2. This result itself will be used as a subroutine to 
design a finite-time algorithm for mixed-integer convex problems in n integer 
and d continuous variables in Subsection 4.3. 

In this section, the problem of interest is (1): 

min{/(x, y) : g l {x, y) < for 1 < i < m, (x, y) £ Z" x R d } 
with a few mild simplifying assumptions. We define the function 
g : R" — > R, x i-> g(x) := min max gi(x,y). 

yGR d l<i<m 

We assume that this minimization in y has a solution for every x £ R n , so as 
to make the function g convex. Let S := {(x, y) £ M. n+d : gi(x, y) < for 1 < 
i < m}. We assume that the function / has a finite spread max{/(x,y) — 
f(x', y') : (x, y), (x' , y') £ S} on S and that we know an upper bound Vf on 
that spread. Observe that, by Lipschitz continuity of / and the assumption 
that we optimize over [—B,B] n , it follows Vf < 2^/nBL. Finally, we assume 
that the partial minimization function: 

(f> : R" -> M U {+oo}, x i y (f)(x) := min{/ (x, y) : (x, y) £ S} 

is convex. As for the function g, this property can be achieved e.g. if for every 
x £ R n for which g(x) < there exists a point y such that (x, y) £ S and 
<j>{x) = f(x,y). 

Our approach is based on the following well-known identity: 

mm{f(x,y) : (x,y) £ S n (Z n x R d )} = min{^(x) : g(x) < 0, x e Z"}. 

For instance, when n = 2, we can use the techniques developed in the pre- 
vious section on <f> to implement the improvement oracle for /. However, we 
cannot presume to know exactly the value of <fi, as it results from a mini- 
mization problem. We merely assume that, for a known accuracy 7 > and 
for every x £ dom</> we can determine a point y x such that (x,y x ) £ S and 
f(x,y x ) — 7 < <p(x) < f(x,y x ). Determining y x can be, on its own, a non- 
trivial optimization problem. Nevertheless, it is a convex problem for which 
we can use the whole machinery of standard Convex Programming (see e.g. 
[NN94, CGT00, Nes04] and references therein.). 
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Since we do not have access to exact values of <f>, we cannot hope for an 
exact oracle for the function <j>, let alone for /. The impact of the accuracy 7 
on the accuracy of the oracle is analyzed in the next subsections. 



4-1 Mixed-integer convex problems with one integer 
variable 

The Algorithm 3 uses as indispensable tools the bisection method for solving 
two types of problems: minimizing a convex function over the integers of an 
interval, and finding, in a given interval, a point that belongs to a level set 
of a convex function. In this subsection, we show how to adapt the bisec- 
tion methods for mixed-integer problems. It is well-known that the bisection 
method is the fastest for minimizing univariate convex functions over a finite 
segment ([Nem94, Chapter 1]). 

Let a, b £ R, a < b, and ip : [a, b] — > R be a convex function to minimize on 
[a, b] and/or on the integers of [a, b], such as the function (f> in the preamble 
of this Section 4 when n = 1. Assume that, for every t <G [a, b], we know a 
number ip(t) € [<*p(t), <p(t) + 7]. In order to simplify the notation, we scale the 
problem so that [a, b] = [0, 1]. The integers of aff{a, b} are scaled to a set of 
points of the form to +rZ for a r > 0. Of course, the spread of the function <p 
does not change, but its Lipschitz constant does, and achieving the accuracy 
7 in its evaluation must be reinterpreted accordingly. 

In the sequel of this section, we fix < Ao < Ai < 1. 

Lemma 4. Under our assumptions, the following statements hold. 

(a) If ip(Xo) < <p(Xi) — 7, then <p(X) > </?(Ao) for all A € [Ai, 1]. 

(b) If<p(X ) > £(Ai)+7, then (p(\) > £(Ai) for all A e [0, A j. 

Proof. We only prove Part (a) as the proof of Part (b) is symmetric. Thus, 
let us assume that ^(Ao) < <^(Ai) — 7. Then there exists < \i < 1 for which 
Ai = fi\ + (1 — /i)A . Convexity of <p allows us to write: 

<?(A ) < £(Ai) - 7 < <p(Ai) < fj,<p(X) + (1 - aOv(^o) < + (1 - A*)<p(Ao), 

implying <^(Aq) < <p(X) as \i > 0. Fig. 8 illustrates the proof graphically. □ 



^(Ao) 








Ao 


Ai 


1 


1 


-vt 






• 


1 



Fig. 8 Lemma 4: the bold line represents a lower bound on ip in Part (a). 
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If one of the conditions in Lemma 4 is satisfied, we can remove from 
the interval [0,1] cither [0,Ao[ or ]Ai,l]. To have a symmetric effect of the 
algorithm in either case, we set Ai := 1 — A , forcing A to be smaller than 
i. In order to recycle our work from iteration to iteration, we choose Ai := 
i(-\/5 — 1), as in the golden search method: if we can eliminate, say, the 
interval ]Ai, 1] from [0, 1], we will have to compute in the next iteration step 
an approximate value of the objective function at AoAi and X\. The latter 
happens to equal Ao when Ai = \ — !)• 

It remains to define a strategy when neither of the conditions in Lemma 4 
is satisfied. In the lemma below, we use the values for Ao, Ai chosen above. 

Lemma 5. Assume that (p{\\) — 7 < <*$(A ) < <^(Ai) + 7. We define: 

Ao+ := (1 — Ao) • Ao + Ao • Ai = 2AoAi, 
A 1-1- := (1 — Ai) ■ Ao + Ai • Ai = 1 — 2AoAi. 

Ifm.m{if(X 0+ ),ip(X 1+ )} < min{(p(A )-7, <^(Ai)-7} ; thenip(t) > min{< y 5(A 0+ ), 
ip(Xi+)} for all t € [0, l]\[Ao, Ai]. Otherwise, it holds that min{<^(Ao) , if(Xi)} < 
mm{ip(t) : t € [0, 1]} + (k - 1)7, where k := ^ w 5.236. 

Proof. The first conclusion follows immediately from Lemma 4. The second 
situation involves a tedious enumeration, summarized in Fig. 9. We assume, 
without loss of generality, that <^(A ) < <^(Ai). The bold lines in Fig. 9 
represent a lower bound on the value of the function ip. We show below 
how this lower bound is constructed and determine its lowest point. In fact, 
this lower bound results from six applications of a simple generic inequality 
(9) that we establish below, before showing how we can particularize it to 
different segments of the interval [0, 1]. 




Let < t < 1 and let u, v e {A , A +, A i+ , Ai}. Suppose that we can write 
v — fit + (1 — /i)u for a fi e ]no, 1] with fi > 0. If we can find constants 
7_ , 7+ > that satisfy 

ip(v) + 7+ > <p(X ) > ip(u) - 7_ 

then we can infer: 
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H<p(t) + (1 - »)(<p{\o) + 7-) > wO) + (1 - M)v(w) > > <^( A o) - 7+. 
and thus: 

^)-^(Ao)>7-- 2±± ^>7-- 2:±± ^. (9) 
A 1 Mo 

1. If t £ ]0, Aq], we can take w := Ai and u := Ao, giving /liq = 1 — ^ = Aq. 



Then 7- = 7+ = 7, and <p(i) - £(A ) > ~7(£ - 1). 



2. If t e ]Ai, 1[, we choose u := Ao and v := Ai, and by symmetry with the 
previous case we obtain fi = X . Now, 7_ = and 7+ = 7, yielding a 
higher bound than in the previous case. 

3. Suppose t £ ]Ao,Ao+]. Then with u := Ai and w := A +, we get fio = 

X x~-x„ = A i> 7- = 7, 7+ = 27, giving as lower bound -7(^7 - 1), which 
is higher than the first one we have obtained. 

4. Symmetrically, let us consider t £ ]Ai + , Ai]. With u := A and v := Ai + , 
we obtain also /j,q = Ai. As 7_ = and 7+ = 27, the lower bound we get 
is larger than the one in the previous item. 

5. Set A' := -|(2A + + 3Ai + ). If t £ ]Ao+,A'], we can use u := A and 
v := Ao+, so that /xo = X y~S\ ° = 5Aq, 7_ = 0, and 7+ = 27. Thus, the 
lower bound is evaluated as — , which is higher than any of the bounds 
we have obtained so far. 

6. Finally, if t £ ]A',A 1+ ], we take u := Xi and v := A 1+ , so that 7_ = 7, 
7+ = 2 7 , and mo = ^ = Hence, we get - 7 (M|+M_i) = _^ 
for the lower bound, just as in the previous item. 

So, the lower bound for <p(t) — <^(A ) on [0, 1] can be estimated as —"f(j^ — 1) ~ 

-4.2367. a 

In the proof of the following proposition, we present an algorithm that 
returns a point x G [0, 1] whose function value ip(x) is close to min{< f o(t) : t £ 
[0,1]}- 

Proposition 3. There exists an algorithm that finds a point x £ [0, 1] for 
which (p{x) — (k — 1)7 < min{ip(t) : t £ [0, 1]} < <p{x) in at most 2 + 

In (^-y--^j / m (Ai)j evaluations of (p, where V v is the spread of if on [0, 1]. 

Proof. We start with the interval [0, 1] and by evaluating (p at Ao and Ai. If 
one of the two conditions in Lemma 4 is satisfied, we can shrink the interval 
by a factor of Ao ~ 38% since it suffices to continue either with the interval 
[0, Ai] or with [Ao, 1]. If not, then Lemma 5 applies: if the first condition stated 
in Lemma 5 is met, then it suffices to continue with the interval [Ao,Ai] so 
as to shrink the starting interval by a factor of 2Ao ~ 76%. Otherwise, any 
x £ [Ao,Ai] satisfies the requirement of the lemma and we can stop the 
algorithm. Therefore, either the algorithm stops or we shrink the starting 
interval by a factor of at least Ao. Iterating this procedure, it follows that — 
if the algorithm does not stop — at every step the length of the remaining 
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interval is at most Ai times the length of the previous interval. Moreover, 
by the choice of A , the function dp is evaluated in two points at the first 
step, and in only one point as from the second step in the algorithm. So, at 
iteration k, we have performed at most 2 + k evaluations of dp. 

By construction, the minimum t* of ip lies in the remaining interval Ik 
of iteration k. Also, the value of <p outside Ik is higher than the best value 
found so far, say doifk)- Finally, the size of Ik is bounded from above by Af. 
Consider now the segment /(A) := (1 — X)t* + A[0, 1], of size A. Observe that 
for every A such that 1 > A > Af, the interval /(A) contains a point that is 
not in Ik- Therefore, 

<p{ik) < max{^(i) : t <= /(A)} < (1 - X)tp(t*) +Amax{^(t) : t G [0,1]} 

< (l-\)<p(t*) + \(V v +<p(t*)). 

Hence <p(tk) — </>(£*) < XV V7 and, by taking A arbitrarily close to A 1; we get 
dpit k ) — <p(t*) < X\V V . If the algorithm does not end prematurely, we need at 

most In {^jr^j / m (^i) iterations to make X\V V smaller than (k — 1)7. 

□ 

Remark 3. If we content ourselves with a coarser precision r\ > (k — 1)7, we 
merely need 0(ln(V v /r))) evaluations of dp. o 

It is now easy to extend this procedure to minimize a convex function 
approximately over the integers of an interval [a, b], or, using our simplifying 
scaling, over (to + rZ) n [0, 1] for given to £ 1 and r > 0. 

Proposition 4. There exists an algorithm that finds a point x e (t + rZ) fl 
[0, 1] for which: 

<p(x) - K7 < min{ (p(i) : i G (t + rZ) n [0, 1]} < (p(x) 

in less than 



mm 



ln((K-l) 7 /V^) 
In(Ai) 



Mr) 
ln(A!) 



evaluations of do, where V v is the spread of <p on [0, 1]. 

Proof. We denote in this proof the points in (to + tZ) as scaled integers. 
To avoid a trivial situation, we assume that [0, 1] contains at least two such 
scaled integers. 

Let us use the approximate bisection method described in the proof of 
Proposition 3 until the remaining interval has a size smaller than r, so that 
it contains at most one scaled integer. Two possibilities arise: either the al- 
gorithm indeed finds such a small interval Ik , or it finishes prematurely, with 
a remaining interval Ik larger than r. 

In the first case, which requires at most 2 + [ln(r)/ ln(Ai)] evaluations 
of <p, we know that Ik contains the continuous minimizcr of ip. Hence, the 
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actual minimizer of <p over (t + tZ) n [0,1] is among at most three scaled 
integers, namely the possible scaled integer in Ik, and, at each side of I k , 
the possible scaled integers that are the closest to Ik- By convexity of ip, the 
best of these three points, say x, satisfies <p(x) — 7 < <p(x) = min{<p(i) : t G 
(t +rZ)n [0,1]}. 

In the second case, we have an interval Ik Q [0, 1] and a point tk that 
fulfill <f(ik) < min{ ip(t) : t £ [0, 1]} + (k — 1)7, which was determined within 
at most 2 + '"^n^/^ evaluations of (p. Consider the two scaled inte- 
gers t— and t+ that are the closest from tk- One of these two points con- 
stitutes an acceptable output for our algorithm. Indeed, suppose first that 
mm{<p(t_),<p(i + )} < <p(i k ) +7. Then: 

min{<p(i_), <p(i + )} < <p(i k ) + 7 < min{<p(i) :te[0, 1]} + k 7 , 

and we are done. Suppose that min{<p(i_), <p(t+)} > <p(ik) +7 and that 
there exists a scaled integer i with y>(f) < mm{(p(t-),ip(t+)}. Without loss 
of generality, let t- G convjt, that is t- = Xt + (1 — A)ffe, with < A < 1. 
We have by convexity of ip: 

<p(t-) < Xp(i) + (1 - \)<p(i k ) < Mi-) + (1 - A)(^(t_) - 7), 

which is a contradiction because A < 1 and <p(t-) — 7 < <p(t-). So, it follows 
that <p(£) > min{y>(£_), <^(f + )} for every t G (t + rZ) n [0, 1], proving the 
statement. □ 

In the the following we extend the above results to the problem min{</?(i) : 
t G [0, 1], g(t) < 0}, where g : [0, 1] — >• R is a convex function with a known 
spread V g . In the case that we have access to exact values of <?, an approach 
for attacking the problem would be the following: we first determine whether 
there exists an element t G [0, 1] with g(t) < 0. If i exists, we determine 
the exact bounds t_ and t+ of the interval {t G [0, l],g(t) < 0}. Then we 
minimize the function / over [f _,£+]. 

The situation where we do not have access to exact values of g or where we 
cannot determine the feasible interval [f _, t+] induces some technical compli- 
cations. We shall not investigate them in this paper, except in the remain- 
ing of this subsection in order to appreciate the modification our method 
needs in that situation: let us assume, that we have only access to a value 
g(t) G [g(t),g(t) +7]. In order to ensure that the constraint g is well-posed 
we make an additional assumption: either {t G [0, 1] : \g(t)\ < 7} is empty, 
or the quantity min{|(;'(i)| : g'(t) G dg(t), \g(t)\ < 7} is non-zero, and even 
reasonably large. This ensures that the (possibly empty) 0-level set of g is 
known with enough accuracy. We denote by 9 > a lower bound on this 
minimum, and for simplicity assume that 9 — 2 N j for a suitable N E N. 

Our strategy proceeds as follows. First we determine whether there exists 
a point t G [0, 1] for which g(t) < by applying the minimization procedure 
described in Proposition 3. If this procedure only returns nonnegative values, 
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we can conclude after at most 2 + [ln((«; — 1)7/1^)/ ln(Ai)] evaluations of g 
that g(t) > — (k — 1)7, in which case we declare that we could not locate any 
feasible point in [0,1]. 

Otherwise, if we find a point i G [0, 1] with g(t) < 0, we continue and 
compute approximate bounds £_ and t + of the interval {t G [0, l],g(t) < 0}. 
For that, we assume g(0),g(l) > 0. By symmetry, we only describe how to 
construct t- such that g{t~) < and g(t- — n) > for an n > reasonably 
small. Note that g(t) < on [t-,t\ by convexity of g. 

In order to compute t- , we adapt the standard bisection method for finding 
a root of a function. Note that the function g might not have any root as 
it might not be continuous. Our adapted method constructs a decreasing 
sequence of intervals [a*,, bk] such that g( a fc) > 0, g(bk) < 0, and fe/c+i — afe+i = 
\(bk — ftfe). If g{dk) > 7: we know that g is positive on [0,0^], and we know 
that there is a root of g on [a k , bk]- Otherwise, if < g{a,k) < 7 and that the 
interval [dfc,6fc] has a length larger or equal to ^. Given the form of 9, wc 
know that k < N. We claim that for every < t < min{0, at - ]} we have 
g(t) > 0, so that we can take 77 := 2^ and t- := b^. Indeed, assume that 
g'(a k ) > 0, then 

g(b k ) > g(b k ) > g(a k ) + g'{a k ){b k - a k ) > -7 + ■ | > 

giving a contradiction, so we must have g'(ak) < —0. We can exclude the 
case where t can only be 0. As claimed, we have 

g(t) > g(a k ) + g'(a k )(t - a k ) > -7 + 0(a k - t) > 

as ^ < a k — t. This takes [ln(^)/ ln(|)] evaluations of g. 

Summarizing this, we just sketched the proof of the following corollary. 

Corollary 3. There exists an algorithm that solves approximately min{<^(£) : 
t e [0,1], g(£) < 0} ; in the sense that it finds, if they exist, three points 
0<£-<x<t+<l with: 

(a) g(t) < g(t) < for every t G [*-,*+], 

(b) ift- > 21, then g(t) > for every t G [0,i_ - 2%], 

(c) if t + < 1 - 2'j, then g(t) > for every t G [£+ + 2^,1], 

(d) <p(x) < min{^(t) : t G [*_,*+], g(t) < 0} + (k - 1)7 



within at most 3 + 
2 + 



ln((/c-l) 7 /V g ) 
ln(Ai) 



ln((«-l) 7 /V y ) 
ln(Ai) 



+ 2 

evaluations of (p. 



Hi/0) 

ln(l/2) 



evaluations of g and at most 



As stressed before above, we assume from now on that we can compute 
exactly the roots of the function g on a given interval, so that the segment 
[£_,£+] in Corollary 3 is precisely our feasible set. This situation occurs e.g. in 
mixed-integer convex optimization with one integer variable when the feasible 
set 5" C M x R d is a polytope. 
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Remark 4- In order to solve problem (1) with one integer variable, we can 
extend Proposition 4 to implement the improvement oracle Oo. K 7- We need 
three assumptions: first, S C [a, b] x R d with a < b; second, / has a finite 
spread on the feasible set; and third we can minimize /(x, y) with (a;, y) G S 
and x fixed up to an accuracy 7. That is, we have access to a value (p(x) G 
\ip(x),ip(x) + 7] with ip(x) := min{/(x, y) : (x,y) G S} being convex. 

Given a feasible query point (x, y) G [a, 6] x R d , we can determine correctly 
that there is no point (x,y) G ((to + tZ) n [0, 1]) x M d for which f(x,y) < 
f(x,y), provided that the output x of our approximate bisection method for 
integers given in Proposition 4 satisfies ip(x) — > f(x,y). Otherwise, we 
can determine a point (x,y) for which f(x,y) < f(x,y) + kj. Note that this 
oracle cannot report a and b simultaneously. o 



4-2 Mixed-integer convex problems with two integer 
variables 

We could use the Mirror-Descent Method in Algorithm 1 to solve the generic 
problem (1) when n = 2 with z 5IMI2 as function V, so that a = 1 
and M = idiam(S') 2 , where diam(S') = max{||z — z'\\ 2 : z,z' G S}. Ac- 
cording to (5), the worst-case number of iterations is bounded by a multiple 
of Ly/M/cr = 0(L diam(5)), where L is the Lipschitz constant of /. As 
Vf < Ldiam(S'), the resulting algorithm would have a worst-case complexity 
of Q{V f ). 

We improve this straightforward approach with a variant of Algorithm 3, 
whose complexity is polynomial in ln(V/). This variant takes into account the 
fact that we do not have access to exact values of the partial minimization 
function (j) defined in the preamble of this section. 

Proposition 5. Suppose that we can determine, for every x G M" with 
9{ x ) < 0, a point y x G R satisfying f(x, y x ) — 7 < min{/(ir, y) : (x, y) G S}. 
Then we can implement the oracle Oo jK7 such that for every (x, y) G S it 
takes a number of evaluations of f that is polynomial in \n(Vf/j). 

Proof. We adapt the algorithm described in the proof of Theorem 2 for 
the function <p(x) := min{/(a;, y) : (x,y) G 5}, which we only know ap- 
proximately. Its available approximation is denoted by <p(x) := f(x, y x ) G 
[<f>(x),<f>(x)+i\. 

Let (x, y) G S be the query point and let us describe the changes that the 
algorithm in Theorem 2 requires. We borrow the notation from the proof of 
Theorem 2. 

The one-dimensional integer minimization problems which arise in the 
course of the algorithm require the use of our approximate bisection method 
for integers in Proposition 4. This bisection procedure detects, if it exists, a 
point x on the line of interest for which (p(x) = f(x, yz) < f(x,y) + kj and 
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we are done. Or it reports correctly that there is no integer x on the line of 
interest with <p(x) < f(x,y). 

In Case 2, wo would need to check whether <f>(z) < f(x, y). In view of our 
accuracy requirement, we only need to check (p(z) < f(x,y) + ivy. 

We also need to verify whether the line H intersects the level set {x e 
R 2 | <p(x) < f(x,y)}. We use the following approximate version: 

"check whether there is a v € conv{z , z} for which cp(v) < f(x, y) + (k — 1)7", 

which can be verified using Proposition 3. If such a point v exists, the convex- 
ity of (j) forbids any w € conv{z, z\} to satisfy (f>(w) < f(x,y), for otherwise: 

4>{z) < 4>(z)+y < max.{<f)(v), 4>(w)}+y < max{0(w), 4>(w)}+y < f(x,y)+ivy, 

a contradiction. Now, if such a point v does not exist, we perform the same 
test on conv{z,2;i}. We can thereby determine correctly which side of z on 
H has an empty intersection with the level set. □ 

Similarly as in Corollary 1, we can extend this oracle into an approximate 
minimization procedure, which solves our optimization problem up to an 
accuracy of ivy, provided that we have at our disposal a point (x, y) € S such 
that f(x,y) is a lower bound on the mixed-integer optimal value. 

Let us now modify our method for finding the fc-th best point for two- 
dimensional problems to problems with two integer and d continuous vari- 
ables. Here, we aim at finding — at least approximately — the fc-th best fiber 
x* k e [-B, B] 2 , so that: 

(x* k ,y* k ) e argmin{/(x, y) :(x,y)eSn ((Z 2 \ {x\, . . . , x R d )} 

for a y k e M d . We set ft, := f(x k ,y k ). The following proposition summarizes 
the necessary extensions of Subsection 3.2. 

Proposition 6. Let k > 2 and let 77 fe _i := {z*, z%_ x } C [-B, B] 2 n Z 2 
be points for which (f>(z*) < f* + iny, g(z*) < when 1 < i < k and such 
that conv{77fc_i} n Z 2 = LJk-\. In a number of approximate evaluations of f 
and gi, . . . ,g m that is polynomial in \n{Vf /y) and k, one can either 

(a) find an integral point z* k G [-B, B] 2 for which <t>{z k ) < f* k ^ + kny, g(z k ) < 
and conv{i7 fc _i, z*} n Z 2 = 77 fe _i U {z* k }, or 

(b) show that there is no integral point z k e \—B, B] 2 for which g(z k ) < 0. 

Proof. If k = 2, we run Algorithm 3 applied to z\ with Line 9 replaced by 
solving mm{{h,y) : y e T k n Z 2 , (h,y) > (h,z*) + 1}, where h £ Z 2 such 
that gcd(/ii, /12) = 1- We also need to use approximate bisection methods 
instead of exact ones. Following the proof of Proposition 5, the oracle finds, 
if it exists, a feasible point z%- Either 4>(z%) < (j>(z*) + ny < f*^ + 2ny < 

fk + 2k 7, or 4>(z*) > 4>{zl) + ny, then </>(£*) < j>(z*) < ft, + ny. Note 
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that, if </>(z 2 ) > <H^i) + K l> we can conclude a posteriori that z* corresponds 
precisely to /[*]. 

For k > 3, we can define the same triangulation as in Figure 7. Replicating 
the observation sketched above, we generate indeed a feasible point z k for 
which j>{z* k ) < /[* fe] + knj. 

Lemma 3 is extended as follows. Suppose that there is an integer point 
x in conv{i7fc_i,5£} \ (II k _ 1 U {z^}). Since <j>(x) < <j>(x) < ffa + knj and 

g(x) < for every x G iTfe-i U {z k }, we have 0(x) < f* k ^ + kn-f and g(x) < 
by convexity. Thus, we can apply Algorithm 4 to find a suitable point z k in 
conv{77 fc _i,^}. □ 



4.3 A finite-time algorithm for mixed-integer convex 
optimization 

In this subsection, we explain how to use the results of the previous subsection 
in order to realize the oracle O a ,8 for a > 0, 6 > in the general case, i.e., 
with n > 3 integer and d continuous variables as in (1). 

Let zeSC [-B, B] n x R d be the query point of the oracle. The oracle 
needs to find a point z e S n (Z n x R d ) for which f(z) < (1 + a)f(z) + 5 (so 
as to report a), or to certify that f(z) < f(z) for every ieSfl (Z rl x R d ) (so 
as to report b) . To design such an oracle we have at our disposal a procedure 
to realize the oracle O a g for any mixed-integer convex minimization problem 
of the kind (1) with n = 2. We propose a finite-time implementation of O a ,s 
with a — and S = n-f. The main idea is to solve the n-dimensional case 
iteratively through the fixing of integer variables. This works as follows. We 
start by solving approximately the relaxation: 

A* 2 := min{/(x,y) : (x,y) £ Sfl (Z 2 x R^+ d )} 

with the techniques developed in the previous subsection. If we can solve the 
partial minimization problems up to an accuracy of 7 < 5/k, we obtain a 
point (u*, &2, £3, . . . , x* n , y*) £ S with u\,u\ E 7L and for which: 

/12 : = f(u*i, U2, xl, ...,<, y*) < f* 2 + K7 

As fi 2 is a lower bound on the mixed-integer optimal value /* , we can make 
our oracle output b if /f 2 — kj > f(z). So, assume that /j* 2 — ^7 < f(z). 

Then we fix Xi :— u* for i = 1,2 and solve (if k > 4; if k = 3, the necessary 
modifications are straightforward) 

A* 234 := min{/0r,i,) : (x,y) e Sn ((fi^fij) x Z 2 x R("- 4 )+ d )}. 
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We obtain a point (u*, . . . , u%, xt,, . . . , x* n , y*) e S with u* e Z for 1 < i < 4 
and for which: 

/l*234 : = f(u*l, ■■■,K,X5,...,X* n ,y*) < /i 2 3 4 + «7 < /* + KJ. 

Now, if /1234 — «7 > /(z), we can make our oracle output b. Thus, we assume 
that /1234 — «7 < f(z) and fix := it* for 1 < z < 4. Iterating this procedure 
we arrive at the subproblem (again, the procedure can easily be modified if 
n is odd): 

mm{f{x, y) : (x, y) e S n ((uj, . . . , <_ 2 ) x Z 2 x 

Let (u*, . . . , u* , t/*) e Z™ x M. d be an approximate optimal solution. If we 
cannot interrupt the algorithm, i.e., if f{u\, . . . , u* , y*) ^ (1 + a)f(z) + kj, 
we replace (u* _ 3 , u^_ 2 ) by the second best point for the corresponding mixed- 
integer convex minimization problem. In view of Proposition 6, the accuracy 
that we can guarantee on the solution is only 2^7, so the criterion to output 
b must be adapted accordingly. Then we proceed with the computation of 
u*) and so on. 

It is straightforward to verify that this approach results in a finite-time 
algorithm for the general case. In the worst case the procedure forces us to 
visit all integral points in [-B, B] n . However, in the course of this procedure 
we always have a feasible solution and a lower bound at our disposal. Once 
the lower bound exceeds the value of a feasible solution we can stop the 
procedure. It is precisely the availability of both, primal and dual information, 
that makes us believe that the entire algorithm is typically much faster than 
enumerating all the integer points in [— B,B] n . 
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